heroku additions - initial test #7678
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
| from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj | ||
|
|
||
|
|
||
| class HerokuError(BaseLLMException): |
There was a problem hiding this comment.
can we move this to a common_utils.py outside of the /chat
make it easier for someone to click on this file and directly see the transformation logic
| # Assuming `raw_response` is an httpx.StreamingResponse object | ||
| for chunk in raw_response.iter_text(): | ||
| streamed_content.append(chunk) | ||
| logging_obj.post_call( |
There was a problem hiding this comment.
let's not call post_call in here - it'll get emitted on every chunk
| stream = optional_params.get("stream", False) | ||
| print_verbose = optional_params.get("print_verbose", False) | ||
|
|
||
| if stream: |
There was a problem hiding this comment.
streaming logic won't enter here - it'll go to 'get_model_response_iterator'
| """ | ||
| Transform the response from the API. | ||
|
|
||
| Handles both standard and streamed responses. |
There was a problem hiding this comment.
no don't handle both in here. we have a separate event hook for streaming -
| ) | ||
|
|
||
| # Call the base method for standard (non-streaming) responses | ||
| return super()._transform_response( |
There was a problem hiding this comment.
if no translation is required, this function doesn't need to be implemented (i see you inherit from openailike config, which should take care of this)
|
Hi @hillarysanders left a few comments, let me know when this pr is ready for review. Ideally it would also include testing - see: https://github.com/BerriAI/litellm/blob/main/tests/llm_translation/test_deepseek_completion.py |
|
@krrishdholakia Thank you so much for the review comments! I will implement them, but I've realized I need to do so in a private repo for now and then submit an updated PR when that's ready to go. Thanks so much for the comments, super helpful <3. I will add you as a reviewer if it's alright with you when the new PR is ready. TYSM!! |
Title
(TODO, still WIP)
Relevant issues
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes
[REQUIRED] Testing - Attach a screenshot of any new tests passing locally
If UI changes, send a screenshot/GIF of working UI fixes